Our "Exchange 2013" server died (because of broken Intel RAID drivers that "Windows" doesn't tell you are broken - "We recommend that you do not use this driver as it is known to have problems with Windows" - unless you try manually to update the driver and select the existing installed version; gee thanks, that's helpful ...).

That was a (considerable ...) pain.  However we still had the "Exchange" databases, so as per the disaster recovery guide we re-built the server (not using an Intel RAID controller!), same name, same configuration (except for the RAID disc, but that wasn't used by "Exchange"), then tried to run "setup /M:RecoverServer /IAcceptExchangeServerLicenseTerms".

And that's where the problems started ...

The installation failed with:

Configuring Microsoft Exchange Server

    Preparing Setup                                           COMPLETED
    Stopping Services                                         COMPLETED
    Copying Exchange Files                                    COMPLETED
    Language Files                                            COMPLETED
    Restoring Services                                        COMPLETED
    Language Configuration                                    COMPLETED
    Mailbox role: Transport service                           FAILED
     The following error was generated when "$error.Clear();
          Install-ExchangeCertificate -services IIS -DomainController $RoleDomai
nController
          if ($RoleIsDatacenter -ne $true -And $RoleIsPartnerHosted -ne $true)
          {
            Install-AuthCertificate -DomainController $RoleDomainController
          }
        " was run: "Microsoft.Exchange.Management.Clients.FormsAuthenticationMar
kPathUnknownSetError: An unexpected error occurred while modifying the forms aut
hentication settings for path /LM/W3SVC/1.  The error returned was 5506.
   at Microsoft.Exchange.Configuration.Tasks.Task.ThrowError(Exception exception
, ErrorCategory errorCategory, Object target, String helpUrl)
   at Microsoft.Exchange.Configuration.Tasks.Task.WriteError(Exception exception
, ErrorCategory category, Object target)
   at Microsoft.Exchange.Management.SystemConfigurationTasks.InstallExchangeCert
ificate.EnableForServices(X509Certificate2 cert, AllowedServices services)
   at Microsoft.Exchange.Management.SystemConfigurationTasks.InstallExchangeCert
ificate.InternalProcessRecord()
   at Microsoft.Exchange.Configuration.Tasks.Task.<ProcessRecord>b__b()
   at Microsoft.Exchange.Configuration.Tasks.Task.InvokeRetryableFunc(String fun
cName, Action func, Boolean terminatePipelineIfFailed)".


The Exchange Server setup operation didn't complete. More details can be found
in ExchangeSetup.log located in the <SystemDrive>:\ExchangeSetupLogs folder.

So it failed trying to install a certificate?!  There's a perfectly good, public CA provided, one already installed on the system and bound in IIS, why on Earth is the moronic "Exchange" installation programme trying to overwrite an existing valid configuration with its' own silly little self-signed certificate?!  Not particularly impressed with the "error handling" being to dump a partial stack trace of a failed PowerShell cmdlet either - that's not how to report errors now is it? ...

And now the server is completely borked.  It was then failing with a whinge about "The language pack bundle could not be found or is corrupt.", but that at least is easily rectified by nuking the "HKLM/SOFTWARE/Microsoft/ExchangeServer/V15/Language Packs" registry key to force them to be re-installed (so why doesn't the installer figure out for itself that they might need reinstalling if a previous installation failed?!  Error handling and recovery 101 ...  And, for that matter, why was the installation of the language packs - which we don't want anyway - corrupt when the installer said it had successfully completed that phase?).

Now it complains that "A Setup failure previously occurred while installing the HubTransport role" and suggests "Either run Setup again for just this role, or remove the role using Control Panel".  But you can't run setup again for just that role, as specifying roles isn't supported for /M:RecoverServer (and if you try and install the role without giving /M:RecoverServer you just get a different irrelevant pile of nonsense errors instead); you can't remove the role using Control Panel because that whinges that a previous setup failed and you now have to run it from the install location.  So that help text was useful wasn't it - might possibly be a good idea for your error message reporting actually to reflect the mode in which the setup programme was run, no?

So the installation failed because of something it shouldn't have been trying to do anyway, and now we can't install and we can't uninstall.  Seriously, you folks have got to learn about error handling and recovery - cheez ...

So just what exactly are we supposed to do now?  The help text is just plain wrong; I would re-build the server (again) but I've no reason to expect that doing so and re-trying the "Exchange" installation would have even slightly different results from it failing this time.  So ...?!

• Edited by TEHM 19 hours 44 minutes ago Large chunks display in italics when using Chrome to edit, but the post itself appears OK when submitted ...

I would call Microsoft support and they can walk you through this step by step.

You don't have to restore the existing server to get things back up however.

You can build a new server with a new name and use database portability and then worry about getting the old server removed from AD and the config later if you want.

https://technet.microsoft.com/en-us/library/dd876926(v=exchg.150).aspx

I would call Microsoft support and they can walk you through this step by step.

You don't have to restore the existing server to get things back up however.

You can build a new server with a new name and use database portability and then worry about getting the old server removed from AD and the config later if you want.

https://technet.microsoft.com/en-us/library/dd876926(v=exchg.150).as

Isn't it annoying when you struggle for ages with a problem, finally get around to posting about it ...

... and then manage to fix it yourself?  Well, so more "Thank *&^% for that" and less "oh isn't that annoying ..." I suppose, however ...

In case anyone else comes across this post looking for a solution, here is what I did (and before anyone jumps in and quips "Oh but that isn't a supported method ...", anyone in this situation probably doesn't really care whether or not it's "supported" - obviously this isn't a supported or recommended fix, just in case anyone was still in doubt - or at least doesn't care nearly as much about that as just getting their d@mn data back):

Firstly, I found from a helpful post here that the particular pile of unhelpful PowerlessShell stack trace I got actually is a synonym for "Oh look, you've already got an SSL (https) binding on your web site and our installation code is too moronic to realise, so we're going to die in an unhelpful way and completely bork your server".  So the first thing to do is check whether you have SSL (https) bindings on your IIS, and particularly whether you've ticked "Require SSL" ("SSL Settings" in the server content view).  If so, unbind, untick, and do whatever you need to to get that off your IIS.

Next thing to do is to stop any "Exchange" services ("services.msc", or however you prefer to get to it) which might have been left running.  Then go and download a copy of the SysInternals Process Explorer (absolutely invaluable) utility; run it, use the binoculars to search for "V15", and kill all the processes which "services" tell you aren't actually running but which obviously are.  Once you've done that, you should be able to go to "%SystemDrive%\Program Files\Microsoft\Exchange Server\" and delete the entire "V15" folder (obviously this will be different if you used an alternative installation location).

Now, fire up "RegEdit.exe" and go to "HKLM\SOFTWARE\Microsoft\ExchangeServer" and delete "v15".  While you're at it, fire up a copy of "notepad.exe" and copy'n'paste the following into it:

REGEDIT4

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\AdminTools]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\CafeRole]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\ClientAccessRole]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\Cmdlet]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\Diagnostics]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\eDiscovery]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\FIP-FS]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\FrontendTransportRole]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\HealthManager]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\HubTransportRole]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\Language]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\MailboxRole]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\Pickup]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\Search]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\ServerComponentStates]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\Setup]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\Transport]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\UnifiedMessagingRole]

Save that somewhere sensible, with the extension ".reg" - something like "C:\ExchangeDevelopersReallyNeedToLearnTheMeaningOfErrorChecking.reg", for example.  Oh, for the terminally paranoid, I've included that as a text format registry file so you can see that it's not doing anything malicious, just creating a bunch of keys.  You will need this file later ...

Now you can re-run your "setup /M:RecoverServer ..." again.

As soon as the prerequisites checking has completed, if you refresh your "RegEdit.exe" you'll see a "v15" key re-appear.  As soon as it has, use the "Import" ("File" menu) option to import the ".reg" file you prepared earlier.  It seems that another useful (that's sarcasm ...) feature of the "Error Recovery Mode" of "Exchange" is that it doesn't create some of the registry keys; an equally helpful feature is that if it tries to refer to a key and fails because it isn't there, it all dies horribly and borks your server ...  More excellent error detection and correction code at work there ...

Incidentally, you probably won't need all the keys in the collection above; it will depend on what roles are being re-installed for a start.  However it does no harm to create that whole bunch anyway.

Now, wait.  The installation will proceed.  However it will probably get stuck on at least one client role (if they're being re-installed), probably somewhere around the 90% mark.  If you leave it, it will fail complaining that it couldn't start some random service or other (another useful feature of the error recovery mode is that it frequently apparently doesn't actually bother starting things it's just installed).  So don't - when you see anything getting to more than about 80% complete, switch to your "services.msc" and start any "Microsoft Exchange ..." services for roles which have now been re-installed.  I.e., if you see a "Microsoft Exchange Unified Messaging" service, and your setup hasn't yet installed "Unified Messaging", don't try to start that service!  However do start anything which looks as though it might just have been installed as part of this setup.

And voila!  You should end up with a re-installed "Exchange 2013" server.

Now you're probably going to have to fix any hang-overs from when your original server died in the first place, starting with probably needing to mount the mailbox database (again, error recovery doesn't seem, in Microsoft's book, to include actually doing anything useful like that ...).  Also use the "Exchange Toolbox" to check that you don't have any stuck queues; send test messages through it all to check all the routing; ...  However presumably you're only doing this so you can get back your users' data onto a nice, shiny, new, "Exchange Server", before dropping a small tactical nuke on the old, re-built one, so checking all the mail flow is possibly unnecessary ...

Anyway, that's what finally did it for me - Hope that helps someone else!

Oh, and if you're anyone from the "Exchange" "development" team in Microsoft and you're reading this thread - be very, very, very, embarrassed about the truly atrocious state of your installation/repair/removal processes ...  And for *&^%s sakes learn the meaning of error detection and recovery!

• Marked as answer by TEHM 14 hours 10 minutes ago